Detection of liveness

ABSTRACT

Detecting a replay attack on a voice biometrics system comprises: receiving a speech signal from a voice source; generating and transmitting an ultrasound signal through a transducer of the device; detecting a reflection of the transmitted ultrasound signal; detecting Doppler shifts in the reflection of the generated ultrasound signal; and identifying whether the received speech signal is indicative of liveness of a speaker based on the detected Doppler shifts. The method further comprises: obtaining information about a position of the device; and adapting the generating and transmitting of the ultrasound signal based on the information about the position of the device.

TECHNICAL FIELD

Embodiments described herein relate to methods and devices for detectingliveness of a speaker. As one example, the liveness detection can beused for detecting a replay attack on a voice biometrics system.

BACKGROUND

Biometrics systems are becoming widely used. In a voice biometricssystem, a user trains the system by providing samples of their speechduring an enrolment phase. In subsequent use, the system is able todiscriminate between the enrolled user and non-registered speakers.Voice biometrics systems can in principle be used to control access to awide range of services and systems.

One way for a malicious party to attempt to defeat a voice biometricssystem is to obtain a recording of the enrolled user's speech, and toplay back the recording in an attempt to impersonate the enrolled userand to gain access to services that are intended to be restricted to theenrolled user.

This is referred to as a replay attack, or as a spoofing attack.

In a facial recognition, or other type of biometrics system, the systemrecognises a characteristic of the user. Again, one way for a maliciousparty to attempt to defeat such a biometrics system is to present thesystem with a photograph or video recording of the enrolled user.

SUMMARY

According to an aspect of the present invention, there is provided amethod of liveness detection in a device. The method comprises:receiving a speech signal from a voice source; generating andtransmitting an ultrasound signal through a transducer of the device;detecting a reflection of the transmitted ultrasound signal; detectingDoppler shifts in the reflection of the generated ultrasound signal; andidentifying whether the received speech signal is indicative of livenessof a speaker based on the detected Doppler shifts. The method furthercomprises: obtaining information about a position of the device; andadapting the generating and transmitting of the ultrasound signal basedon the information about the position of the device.

According to another aspect of the present invention, there is provideda system for liveness detection, the system being configured forperforming the method of the first aspect.

According to another aspect of the present invention, there is provideda device comprising such a system. The device may comprise a mobiletelephone, an audio player, a video player, a mobile computing platform,a games device, a remote controller device, a toy, a machine, or a homeautomation controller or a domestic appliance.

According to another aspect of the present invention, there is provideda computer program product, comprising a computer-readable tangiblemedium, and instructions for performing a method according to the firstaspect.

According to another aspect of the present invention, there is provideda non-transitory computer readable storage medium havingcomputer-executable instructions stored thereon that, when executed byprocessor circuitry, cause the processor circuitry to perform a methodaccording to the first aspect. According to a further aspect, there isprovided a device comprising the non-transitory computer readablestorage medium. The device may comprise a mobile telephone, an audioplayer, a video player, a mobile computing platform, a games device, aremote controller device, a toy, a machine, or a home automationcontroller or a domestic appliance.

BRIEF DESCRIPTION OF DRAWINGS

For a better understanding of the present invention, and to show how itmay be put into effect, reference will now be made to the accompanyingdrawings, in which: —

FIG. 1 illustrates a smartphone.

FIG. 2 is a schematic diagram, illustrating the form of the smartphone.

FIG. 3 illustrates a situation in which a replay attack is beingperformed;

FIG. 4 is a flow chart illustrating a method of detecting liveness;

FIG. 5 illustrates a speech processing system, including a system fordetecting liveness;

FIG. 6 is a flow chart illustrating a part of the method of detectingliveness;

FIGS. 7a, 7b, and 7c illustrate various possible uses of smartphones;

FIG. 8 is a flow chart illustrating a part of the method of detectingliveness;

FIG. 9 is a flow chart illustrating a part of the method of detectingliveness;

FIG. 10 is a block diagram, illustrating a part of the system fordetecting liveness; and

FIG. 11 illustrates results of the method of detecting liveness.

DETAILED DESCRIPTION OF EMBODIMENTS

The description below sets forth example embodiments according to thisdisclosure. Further example embodiments and implementations will beapparent to those having ordinary skill in the art. Further, thosehaving ordinary skill in the art will recognize that various equivalenttechniques may be applied in lieu of, or in conjunction with, theembodiments discussed below, and all such equivalents should be deemedas being encompassed by the present disclosure.

One example of the invention is illustrated with reference to its use ina smartphone, by way of example, though it will be appreciated that itmay be implemented in any suitable device, as described in more detailbelow.

FIG. 1 illustrates a smartphone 10, having a microphone 12 for detectingambient sounds. In normal use, the microphone is of course used fordetecting the speech of a user who is holding the smartphone 10.

The smartphone 10 also has two loudspeakers 14, 16. The firstloudspeaker 14 is located at the top of the smartphone 10, when it isheld in its normal operating position for making a voice call, and isused for playing the sounds that are received from the remote party tothe call.

The second loudspeaker 16 is located at the bottom of the smartphone 10,and is used for playing back media content from local or remote sources.Thus, the second loudspeaker 16 is used for playing back music that isstored on the smartphone 10 or sounds associated with videos that arebeing accessed over the internet.

The illustrated smartphone 10 also has two additional microphones 12 a,12 b. The additional microphones, if present in the device, may beprovided at any suitable location. In this illustrated device, onemicrophone 12 a is located at the top end of the front of the device,while another microphone 12 b is located at the top end of the side ofthe device.

FIG. 2 is a schematic diagram, illustrating the form of the smartphone10.

Specifically, FIG. 2 shows various interconnected components of thesmartphone 10. It will be appreciated that the smartphone 10 will inpractice contain many other components, but the following description issufficient for an understanding of the present invention.

Thus, FIG. 2 shows the microphone 12 mentioned above. In this particularillustrated embodiment, the smartphone 10 is provided with multiplemicrophones 12, 12 a, 12 b, etc. FIG. 2 also shows the loudspeakers 14,16.

FIG. 2 also shows a memory 18, which may in practice be provided as asingle component or as multiple components. The memory 18 is providedfor storing data and program instructions.

FIG. 2 also shows a processor 20, which again may in practice beprovided as a single component or as multiple components. For example,one component of the processor 20 may be an applications processor ofthe smartphone 10.

FIG. 2 also shows a transceiver 22, which is provided for allowing thesmartphone 10 to communicate with external networks. For example, thetransceiver 22 may include circuitry for establishing an internetconnection over a WiFi local area network and/or over a cellularnetwork.

FIG. 2 also shows audio processing circuitry 24, for performingoperations on the audio signals detected by the microphone 12 asrequired. For example, the audio processing circuitry 24 may filter theaudio signals or perform other signal processing operations.

The audio signal processing circuitry is also able to generate audiosignals for playback through the loudspeakers 14, 16, as discussed inmore detail below.

FIG. 2 also shows that the smartphone 10 may include one or more sensors26. In certain embodiments, the sensor(s) may include any combination ofthe following: gyroscopes, accelerometers, proximity sensors, lightlevel sensors, touch sensors, and a camera.

In this illustrated embodiment, the smartphone 10 is provided with voicebiometric functionality, and with control functionality. Thus, thesmartphone 10 is able to perform various functions in response to spokencommands from an enrolled user. The biometric functionality is able todistinguish between spoken commands from the enrolled user, and the samecommands when spoken by a different person. Thus, certain embodiments ofthe invention relate to operation of a smartphone or another portableelectronic device with some sort of voice operability, for example atablet or laptop computer, a games console, a home control system, ahome entertainment system, an in-vehicle entertainment system, adomestic appliance, or the like, in which the voice biometricfunctionality is performed in the device that is intended to carry outthe spoken command. Certain other embodiments relate to systems in whichthe voice biometric functionality is performed on a smartphone or otherdevice, which then transmits the commands to a separate device if thevoice biometric functionality is able to confirm that the speaker wasthe enrolled user.

In some embodiments, while voice biometric functionality is performed onthe smartphone 10 or other device that is located close to the user, thespoken commands are transmitted using the transceiver 22 to a remotespeech recognition system, which determines the meaning of the spokencommands. For example, the speech recognition system may be located onone or more remote server in a cloud computing environment. Signalsbased on the meaning of the spoken commands are then returned to thesmartphone 10 or other local device. In other embodiments, the speechrecognition system is also located on the device 10.

One attempt to deceive a voice biometric system is to play a recordingof an enrolled user's voice in a so-called replay or spoof attack.

FIG. 3 shows an example of a situation in which a replay attack is beingperformed. Thus, in FIG. 3, the smartphone 10 is provided with voicebiometric functionality. In this example, the smartphone 10 is in thepossession, at least temporarily, of an attacker, who has anothersmartphone 30. The smartphone 30 has been used to record the voice ofthe enrolled user of the smartphone 10. The smartphone 30 is broughtclose to the microphone inlet 12 of the smartphone 10, and the recordingof the enrolled user's voice is played back. If the voice biometricsystem is unable to determine that the enrolled user's voice that itrecognises is a recording, the attacker will gain access to one or moreservices that are intended to be accessible only by the enrolled user.

At the same time, or separately, when the smartphone 10 is provided witha camera-based biometric functionality, such as a facial recognitionsystem, an attacker may use the display of the smartphone 30 to show aphoto or video of the enrolled user, in an attempt to defeat the facialrecognition system.

Embodiments described herein therefore attempt to perform livenessdetection, for example detecting the presence of a person speaking anyvoice sounds that are detected.

FIG. 4 is a flow chart, illustrating a method of liveness detection, forexample for use in a biometrics system, and in this illustrated exampleused for detecting a replay attack on a voice biometrics system, andFIG. 5 is a block diagram illustrating functional blocks in one exampleof a speech processing system that includes the voice biometrics system.

Specifically, in step 50 in the method of FIG. 4, a signal is receivedon an input 70 of the system shown in FIG. 5. Thus, the input 70 may beconnected to the microphone 12 shown in FIG. 1 or the multiplemicrophones 12, 12 a, 12 b, etc shown in FIG. 2.

The received signal is passed to a voice activity detector (VAD) 72,which detects when the received signal contains speech.

The received signal is also passed to a keyword detection block 74. Ifit is determined by the voice activity detector 72 that the receivedsignal contains speech, the keyword detection block 74 is activated, andit acts to detect the presence of a predetermined keyword in thedetected speech. For example, the speech processing system of asmartphone might as a default operate in a low power mode, reflectingthe fact that speech processing will be required for only a smallfraction of the operating life of the device. The speech processingsystem may then be taken out of the low-power mode by the user utteringthe predetermined keyword or phrase, such as “Hello phone”.

The received signal is also passed to a speaker recognition block 76. Ifit is determined by the keyword detection block 74 that thepredetermined keyword is present in the detected speech, the speakerrecognition block 76 then attempts to determine whether the person whouttered the predetermined keyword is the registered user of the deviceand/or of a particular application on the device. Suitable biometrictechniques are known for determining whether the speaker of the speechthat is present in the received signal is the registered user.

If it is determined by the speaker recognition block 76 that the personwho uttered the predetermined keyword is the registered user of thedevice and/or of the particular application on the device, then thereceived signal is passed to a speech processing block 78, which may bepresent on the device or may be located remotely, in the cloud. Thespeech processing block 78 then determines the content of the speech. Ifthe speech contains a command, for example, then the speech processingblock 78 generates a suitable signal for causing that command to beacted upon.

The system shown in FIG. 5 includes a mechanism for performing livenessdetection, and hence for detecting whether the received signalcontaining speech has originated from a replay attack, as illustrated inFIG. 3.

Thus, in step 52 of the method shown in FIG. 4, an ultrasound signal isgenerated and transmitted, by the ultrasound generate and transmit block80 shown in FIG. 5. The ultrasound transmit block 80 may operate at alltimes. In other embodiments, the ultrasound transmit block 80 operatesonly when it receives an enable signal on its input 82. The enablesignal may be generated, for example, when the voice activity detector72 determines that the received signal contains speech, or when thekeyword detection block 74 detects the presence of the predeterminedkeyword, or when the speaker recognition block 76 starts to perform abiometric technique to determine whether the person who uttered thepredetermined keyword is the registered user.

The ultrasound signal may be a single tone sine wave, or otherconfigurations may be used, for example a chirp signal. The frequency ofthe ultrasound signal may be selected to be relatively close to 20 kHzfor transmittability reasons, while being high enough to ensure that itis not audible.

In step 54 of the method shown in FIG. 4, a reflection of the generatedultrasound signal is detected.

In the system shown in FIG. 5, a signal is received on an input 84, andpassed to an ultrasound detection block 86. For example, the input 84may be connected to one or more of the multiple microphones 12, 12 a, 12b, etc shown in FIG. 2, to receive any signal detected thereby.

The received signal is passed to the ultrasound detection block 86,which may for example comprise one or more filter for selecting signalshaving a frequency that is close to the frequency of the ultrasoundsignal transmitted by the ultrasound transmit block 80. Reflectedultrasound signals may be Doppler shifted in their frequency, but theDoppler shifts are unlikely to be much more than 100 Hz, and so theultrasound detection block 86 may comprise a filter for selectingsignals having a frequency that is within 100 Hz of the frequency of theultrasound signal transmitted by the ultrasound transmit block 80.

In step 56 of the method shown in FIG. 4, the received ultrasound signaldetected by the ultrasound detection block 86 is passed to a Dopplerdetect block 88, to detect Doppler shifts in the reflection of thegenerated ultrasound signal. Thus, the received reflected ultrasoundsignal is compared with the generated ultrasound signal to identifyfrequency shifts in the reflected signal that are caused by reflectionsoff a moving surface, such as the face, and in particular the lips, of aperson who is speaking to generate the detected speech signal.

In step 58 of the method shown in FIG. 4, it is determined based on thedetected Doppler shifts whether these Doppler shifts provide goodevidence for the liveness of a person generating the detected speech.

In the illustrated embodiment shown in FIG. 5, the output of the Dopplerdetect block 88 is applied to one input of a correlation block 90. Thereceived audio signal on the input 70 is applied to another input of thecorrelation block 90. In an alternative embodiment, a signal generatedby the voice activity detect block 72 is applied to the other input ofthe correlation block 90. The output of the correlation block 90 isapplied to a determination block 92 shown in FIG. 5.

If it is found by the correlation block 90 that there is a correlationbetween time periods in which Doppler shifts are detected in thereflection of the generated ultrasound signal, and time periods in whichspeech content is identified in the received speech signal, thisindicates that the detected speech is generated by a live person movingtheir lips to generate the sound. If the degree of correlation is low,one possible reason for this may be that the detected speech is notgenerated by a live person moving their lips to generate the sound. Onepossible cause of this is that the detected speech is in fact generatedby a replay attack.

Therefore, the determination block 92 produces an output signal thatcontains information about the liveness of the speaker, and hence aboutthe likelihood that the detected speech was generated by a replayattack. This output signal is applied, in this illustrated embodiment,to the speaker recognition block 76, which is performing one or morevoice biometrics process to determine whether the speaker is theregistered user of the device. The speaker recognition block 76 can thenuse the output signal as one of several factors that it uses todetermine whether the speaker is in fact the registered user of thedevice. For example, there may be one or more factors which indicatewhether the detected speech is the speech of the registered user, andone or more factors which indicate whether the detected speech may haveresulted from a replay attack.

In other examples, the liveness detection can be used for otherpurposes, for example for detecting an attempt to defeat a facialrecognition system by presenting a still or moving image of an enrolleduser.

As discussed in more detail below, the purpose of generating theultrasound signal is to detect the movement of a speaker's face, and inparticular the lips, while speaking. For this to operate successfully,it is advantageous that the ultrasound signal may be varied depending oninformation about the use of the device.

Thus, as described above, step 52 of the process shown in FIG. 4involves generating and transmitting the ultrasound signal.

FIG. 6 is a flow chart, giving more detail about this step, in someembodiments. Specifically, in step 110 of the method, the system obtainsinformation about a position of the device 10.

For example, obtaining information about a position of the device maycomprise obtaining information about an orientation of the device.Information about the orientation of the device may for example beobtained from gyroscopes and/or accelerometers provided as sensors 26 inthe device 10.

As one alternative, obtaining information about a position of the devicemay comprise obtaining information about a distance of the device fromthe voice source. Information about a distance of the device from thevoice source may for example be obtained by detecting the levels ofsignals generated by the microphones 12, 12 a, 12 b. For example, ahigher signal level from one microphone may indicate that the voicesource is closer to that microphone than to one or more othermicrophone.

As another alternative, obtaining information about a position of thedevice may comprise obtaining information about a position of the devicerelative to a supposed speaker. Information about the position of thedevice relative to a supposed speaker may for example be obtained fromone or more proximity sensor provided as a sensor 26 in the device 10.Information about the position of the device relative to a supposedspeaker may also be obtained from one or more light level sensorprovided as a sensor 26 in the device 10. Information about the positionof the device relative to a supposed speaker may also be obtained fromone or more touch sensor provided as a sensor 26 in the device 10,indicating how the device 10 is being held by a user. Information aboutthe position of the device relative to a supposed speaker may also beobtained from a camera provided as a sensor 26 in the device 10, whichcan track the position of a user's face relative to the device 10.

Then, in step 112, the method involves adapting the generating andtransmitting of the ultrasound signal based on the information about theposition of the device.

Adapting the generating and transmitting of the ultrasound signal mayfor example comprise adjusting a transmit power of the ultrasoundsignal. As another example, when the device has multiple transducers 14,16, adapting the generating and transmitting of the ultrasound signalmay comprise selecting the one or more transducer in which theultrasound signal is generated, with the intention that the ultrasoundsignal should be generated from a transducer that is close to the user'smouth in order to be able to detect movement of the user's lips.

For example, obtaining information about a position of the device maycomprise obtaining information about a distance of the device from thevoice source, and adapting the generating and transmitting of theultrasound signal may comprise adjusting a transmit power of theultrasound signal, with a higher power being used when the device isfurther from the voice source, at least for distances below a certainlimit. This allows the device to generate ultrasound signals thatproduce clearly detectable reflections, without risking transmittingultrasound energy when the device is close to the user's ear.

As another example, obtaining information about a position of the devicemay comprise obtaining information as to which of multiple loudspeakertransducers is closest to the voice source (for example based on signallevels at microphones placed located close to those transducers), andadapting the generating and transmitting of the ultrasound signal maycomprise transmitting the ultrasound signal mainly or entirely from thattransducer. This allows the device to generate ultrasound signals fromthe transducer that is closest to the sound source, and thereby increasethe chance of detecting usable reflection signals.

Other possibilities relate to specific ways in which speakers may usethe device.

Thus, for example, when the device 10 is a mobile phone comprising atleast a first transducer 16 at a lower end of the device and a secondtransducer 14 at an upper end of the device, adapting the generating andtransmitting of the ultrasound signal based on the information about theposition of the device may comprise transmitting the ultrasound signalat a relatively low power from the first transducer 16 if theinformation about the position of the device indicates that the device10 is being used in a close talk mode. Close talk will be understood asa use of a phone where the phone is positioned adjacent the side of auser's face, and where communication is using the close-range earpiecespeaker, e.g. as with a “traditional” phone handset positioning. Forexample, the ultrasound signal may be transmitted at a level of 70-90 dBSPL at 1 cm in this mode.

The information about the position of the device may be considered toindicate that the device is being used in a close talk mode if, forexample, accelerometers indicate that the device 10 is in an uprightposition, and proximity sensors detect that the device 10 is being heldclose to a surface that might be a user's face 120, as shown in FIG. 7a.

More generally, adapting the generating and transmitting of theultrasound signal based on the information about the position of thedevice may comprise transmitting the ultrasound signal from the secondtransducer if the information about the position of the device indicatesthat the device is being used in a generally vertical orientation.

As another example, when the device 10 is a mobile phone comprising atleast a first transducer 16 at a lower end of the device and a secondtransducer 14 at an upper end of the device, adapting the generating andtransmitting of the ultrasound signal based on the information about theposition of the device may comprise transmitting the ultrasound signalat a relatively high power from the transducer 16 at the lower end ofthe device, if the information about the position of the deviceindicates that the device 10 may be being held by the user in front oftheir face 130, with the lower microphone 12 pointing towards them, i.e.in a “pizza slice” version of a near talk mode, as shown in FIG. 7 b.

Near-talk mode will be understood as where a phone is positioned infront of the user's face, and where use may be made of near-fieldloudspeakers and microphones. This position may be suitable for thepurposes of a video call, e.g. using software products such as Skype™from Microsoft or FaceTime™ from Apple. “Pizza slice” mode will beunderstood as a variation of near-talk mode, but where the phone is heldin a relatively horizontal position (such that a microphone positionedat the lower end of the phone faces the user directly).

For example, the ultrasound signal may be transmitted at a level of90-110 dB SPL at 1 cm in this mode.

The information about the position of the device may be considered toindicate that the device is being used in a “pizza slice” mode if, forexample, accelerometers indicate that the device is in a horizontalposition, and the signal level detected by the microphone 12 is higherthan the signal level detected by the microphones 12 a, 12 b.

More generally, adapting the generating and transmitting of theultrasound signal based on the information about the position of thedevice may comprise transmitting the ultrasound signal from the firsttransducer if the information about the position of the device indicatesthat the device is being used in a generally horizontal orientation.

In the variant of the near talk mode, in which the device is held by theuser in front of their face, for example so that they can see the screenon the device while speaking, adapting the generating and transmittingof the ultrasound signal based on the information about the position ofthe device may comprise transmitting the ultrasound signal at arelatively high power from the transducer 14 at the upper end of thedevice, or from transducers at both ends of the device.

As another example, adapting the generating and transmitting of theultrasound signal based on the information about the position of thedevice may comprise preventing transmission of the ultrasound signal ifthe information about the position of the device indicates that thedevice is being used in a far field mode, for example with the device 10being placed on a surface 140 some distance from the user 142, as shownin FIG. 7c . In this example, the information about the position of thedevice may indicate that the device is located more than a thresholddistance (for example 50 cm) from the source of the sound.

This is because it may be determined that detecting the movement of aspeaker's lips is only reliable enough for use when the indications arethat the device may be being held close to the user's face.

As shown in FIG. 5, and as described above, the output of the Dopplerdetect block 88 is applied to one input of a correlation block 90. Thereceived audio signal on the input 70 is applied to another input of thecorrelation block 90. The correlation block 90 determines whether thereis a correlation between time periods in which Doppler shifts aredetected in the reflection of the generated ultrasound signal, andperiods in which there is speech.

The aim is to confirm that any Doppler shifts that are detected in thereceived reflection of the generated ultrasound signal do result fromfacial movements of a speaker, and are not the result of spuriousreflections from other moving objects.

FIG. 8 is a flow chart, illustrating a method performed in thecorrelation block 90.

First, it is determined in step 150, whether the detected Doppler shiftscorrespond to a general speech articulation rate. The articulation rateis the rate at which syllables are produced during speech, and it hasbeen found that, for most speech, a typical articulation rate is in therange of 4-10 Hz. The facial movements of the speaker (for examplemovements of the speaker's lips, cheeks, and nostrils) will typicallyoccur at the same rate. Thus, in step 150, it is determined whether thedetected Doppler shifts correspond to facial movements at a frequency inthe range of 4-10 Hz.

In step 152, it is determined whether the detected Doppler shiftscorrespond to an articulation rate of the current speech.

Thus, the articulation rate of the speech contained in the receivedaudio signal is extracted in the correlation block 90. It is thendetermined whether the detected Doppler shifts correspond to facialmovements at a frequency that corresponds to that extracted articulationrate.

If it is determined that the detected Doppler shifts correspond tofacial movements at a frequency that corresponds to that extractedarticulation rate, this can be taken as good evidence of liveness.

In a further possible step, in step 154 of the method shown in FIG. 8,it is determined whether there is a correlation between detected Dopplershifts in the reflection of the generated ultrasound signal, and speechcontent of the received speech signal.

It is recognised that one issue with using ultrasound as describedherein, is that there may be interfering sources of ambient ultrasoundnoise.

Therefore, FIG. 9 is a flow chart, illustrating one method performed inthe Doppler detect block 88 and correlation block 90.

Specifically, in step 170, a level of ambient ultrasound noise ismonitored. Then, in step 172, the operation of the voice biometricssystem is adjusted based on the levels of the reflected ultrasound andmonitored ambient ultrasound noise.

FIG. 10 is a block diagram, illustrating schematically the operation ofthe Doppler detect block 88 and correlation block 90. FIG. 11illustrates signals obtained at different stages of the operation.

Specifically, the signal from one or microphone 12 is passed to a lowpass filter 180, for isolating the audio frequency components (forexample, below 20 kHz) of the detected signal. The resulting audiosignal, in one example, is shown in FIG. 11(a).

The signal level of the audio signal is found in a block 182 that findsthe absolute value of the signal. The resulting envelope signal, in thesame example, is shown in FIG. 11(b).

The signal from the one or microphone 12 is also passed to a high passfilter 184, for isolating the ultrasound components (for example, above20 kHz) of the detected signal. This may contain the wanted reflectionof the generated ultrasound signal, but may also contain interferingambient ultrasound noise.

The level of the ultrasound signal is determined by a level detector186.

The ultrasound signal is then passed to a demodulation block 188, whereit is downconverted to the audio band, and any Doppler shiftedreflections are found. This is achieved by mixing the receivedultrasound signal with the ultrasound signal that was generated andtransmitted. The received ultrasound signal can be passed through a bandpass filter before downconversion if required, in order to remove otherultrasound signals not originating from the transmitted signal. Inaddition, the output of the mixing step can be low-pass filtered.

The resulting signal, in one example, is shown in FIG. 11(c).

The signal level of the Doppler shifted reflected signal is found in ablock 190 that finds the absolute value of the signal.

It can thus be seen from FIG. 11 that there is a correlation between thedetected Doppler shifts in the reflection of the generated ultrasoundsignal, and speech content of the received speech signal.

In order to obtain a robust result, a correlation operation isperformed, as shown at block 192 of FIG. 10.

However, before performing the correlation, it is noted that, while theaudio signal is effectively the result of the facial movements of thespeaker, the Doppler shifts in the reflected ultrasound signal willresult from the velocity of the facial movements. Therefore, in someembodiments, either the audio signal is differentiated (for example bypassing through a block 194 in the form of a band pass filter with apass-band of, say, 10-200 Hz, an envelope block, or a differentiator),or the ultrasound signal is integrated (for example by passing through ablock 196 in the form of a leaky integrator or a band pass filter with apass-band of, say, 10-200 Hz).

The correlator 192 then performs a frame-by-frame cross correlation onthe signals. If the correlation result Rxy is above a threshold then itis determined that there is enough of a correlation, between thedetected Doppler shifts and the speech content of the received speechsignal, to conclude that there is evidence of a live speaker, and hencethat the speech may not result from a replay attack. If there is notgood evidence of liveness of a speaker, this may be an indication thatthe received speech signal may be a product of a replay attack.

The operation of the system may be adjusted, based on a level of thereflected ultrasound and the monitored ambient ultrasound noise, asdetected by the level detector 186.

For example, the reliance that is placed on the determination as towhether the received speech signal may be the result of a replay attackmay be adjusted, based on the level of the monitored ambient ultrasoundnoise. The determination, as to whether the received speech signal maybe the result of a replay attack, will typically be made based on morethan one factor. It is recognised that the presence of large ambientultrasound signals will impact on the reliability of this system, and sothe reliance that is placed on the determination may be reduced, as thelevel of the monitored ambient ultrasound noise increases. Morespecifically, if the level of the monitored ambient ultrasound noiseexceeds a first threshold level, the result of the correlation may beignored completely, or the correlation may not be performed.

For lower levels of interference, the adjustment of the operation of thesystem may involve adapting the threshold correlation value that is usedin determining whether there is enough of a correlation, between thedetected Doppler shifts and the speech content of the received speechsignal, to conclude that there is evidence of a live speaker.Specifically, for low levels of ultrasound interference, a highthreshold correlation value can be used. For somewhat higher levels ofultrasound interference (still below the first threshold mentionedabove), lower threshold correlation values can be used, to take accountof the fact that the presence of interference will automatically reducethe correlation values obtained from the correlator 192.

The skilled person will recognise that some aspects of theabove-described apparatus and methods may be embodied as processorcontrol code, for example on a non-volatile carrier medium such as adisk, CD- or DVD-ROM, programmed memory such as read only memory(Firmware), or on a data carrier such as an optical or electrical signalcarrier. For many applications embodiments of the invention will beimplemented on a DSP (Digital Signal Processor), ASIC (ApplicationSpecific Integrated Circuit) or FPGA (Field Programmable Gate Array).Thus the code may comprise conventional program code or microcode or,for example code for setting up or controlling an ASIC or FPGA. The codemay also comprise code for dynamically configuring re-configurableapparatus such as re-programmable logic gate arrays. Similarly the codemay comprise code for a hardware description language such as Verilog™or VHDL (Very high speed integrated circuit Hardware DescriptionLanguage). As the skilled person will appreciate, the code may bedistributed between a plurality of coupled components in communicationwith one another. Where appropriate, the embodiments may also beimplemented using code running on a field-(re)programmable analoguearray or similar device in order to configure analogue hardware.

Note that as used herein the term module shall be used to refer to afunctional unit or block which may be implemented at least partly bydedicated hardware components such as custom defined circuitry and/or atleast partly be implemented by one or more software processors orappropriate code running on a suitable general purpose processor or thelike. A module may itself comprise other modules or functional units. Amodule may be provided by multiple components or sub-modules which neednot be co-located and could be provided on different integrated circuitsand/or running on different processors.

Embodiments may be implemented in a host device, especially a portableand/or battery powered host device such as a mobile computing device forexample a laptop or tablet computer, a games console, a remote controldevice, a home automation controller or a domestic appliance including adomestic temperature or lighting control system, a toy, a machine suchas a robot, an audio player, a video player, or a mobile telephone forexample a smartphone.

It should be noted that the above-mentioned embodiments illustraterather than limit the invention, and that those skilled in the art willbe able to design many alternative embodiments without departing fromthe scope of the appended claims. The word “comprising” does not excludethe presence of elements or steps other than those listed in a claim,“a” or “an” does not exclude a plurality, and a single feature or otherunit may fulfil the functions of several units recited in the claims.Any reference numerals or labels in the claims shall not be construed soas to limit their scope.

1. A method of liveness detection in a device, the method comprising:receiving a speech signal from a voice source; generating andtransmitting an ultrasound signal through a transducer of the device;detecting a reflection of the transmitted ultrasound signal; detectingDoppler shifts in the reflection of the generated ultrasound signal; andidentifying whether the received speech signal is indicative of livenessof a speaker based on the detected Doppler shifts, and the methodfurther comprising: obtaining information about a position of thedevice; and adapting the generating and transmitting of the ultrasoundsignal based on the information about the position of the device.
 2. Amethod according to claim 1, wherein adapting the generating andtransmitting of the ultrasound signal comprises: adjusting a transmitpower of the ultrasound signal.
 3. A method according to claim 1,wherein the device has multiple transducers, and wherein adapting thegenerating and transmitting of the ultrasound signal comprises:selecting the transducer in which the ultrasound signal is generated. 4.A method according to claim 1, wherein obtaining information about aposition of the device comprises obtaining information about anorientation of the device.
 5. A method according to claim 1, whereinobtaining information about a position of the device comprises obtaininginformation about a distance of the device from the voice source.
 6. Amethod according to claim 1, wherein the device is a mobile phonecomprising at least a first transducer at a lower end of the device anda second transducer at an upper end of the device, and wherein adaptingthe generating and transmitting of the ultrasound signal based on theinformation about the position of the device comprises: transmitting theultrasound signal from the first transducer at an intensity in the rangeof 70-90 dB SPL at 1 cm if the information about the position of thedevice indicates that the device is being used in a close talk mode. 7.A method according to claim 1, wherein the device is a mobile phonecomprising at least a first transducer at a lower end of the device anda second transducer at an upper end of the device, and wherein adaptingthe generating and transmitting of the ultrasound signal based on theinformation about the position of the device comprises: transmitting theultrasound signal at an intensity in the range of 90-110 dB SPL at 1 cmif the information about the position of the device indicates that thedevice is being used in a near talk mode.
 8. A method according to claim6, wherein adapting the generating and transmitting of the ultrasoundsignal based on the information about the position of the devicecomprises: transmitting the ultrasound signal from the first transducerif the information about the position of the device indicates that thedevice is being used in a generally horizontal orientation.
 9. A methodaccording to claim 6, wherein adapting the generating and transmittingof the ultrasound signal based on the information about the position ofthe device comprises: transmitting the ultrasound signal from the secondtransducer if the information about the position of the device indicatesthat the device is being used in a generally vertical orientation.
 10. Amethod according to claim 1, wherein adapting the generating andtransmitting of the ultrasound signal based on the information about theposition of the device comprises: preventing transmission of theultrasound signal if the information about the position of the deviceindicates that the device is being used in a far talk mode.
 11. A methodaccording to claim 5, wherein adapting the generating and transmittingof the ultrasound signal comprises adjusting a transmit power of theultrasound signal, with a higher power being used when the device isfurther from the voice source, for distances below a predeterminedmaximum distance.
 12. A method according to claim 1, wherein obtaininginformation about a position of the device comprises obtaininginformation as to which of multiple loudspeaker transducers is closestto the voice source, and adapting the generating and transmitting of theultrasound signal comprises transmitting the ultrasound signal primarilyor entirely from that loudspeaker.
 13. A method according claim 1,comprising obtaining information about the position of the device fromone or more of the following: gyroscopes, accelerometers, proximitysensors, light level sensors, touch sensors, sound level sensors, and acamera.
 14. A method according to claim 1, for use in a voice biometricssystem, wherein identifying whether the received speech signal isindicative of liveness comprises determining whether the received speechsignal may be a product of a replay attack.
 15. A system for livenessdetection in a device, the system comprising: at least one microphoneinput, for receiving an audio signal from a microphone; and at least onetransducer output, for transmitting a signal to an ultrasound, and thesystem being configured for: receiving a speech signal from the at leastone microphone input; generating a control signal through the transduceroutput, for transmitting an ultrasound signal through a transducer ofthe device; detecting a reflection of the transmitted ultrasound signal;detecting Doppler shifts in the reflection of the generated ultrasoundsignal; and identifying whether the received speech signal is indicativeof liveness of a speaker based on the detected Doppler shifts, and themethod further comprising: obtaining information about a position of thedevice; and adapting the generating and transmitting of the ultrasoundsignal based on the information about the position of the device.
 16. Anon-transitory computer readable storage medium havingcomputer-executable instructions stored thereon that, when executed byprocessor circuitry, cause the processor circuitry to perform a methodaccording to claim 1.